METHOD AND SYSTEM FOR ACCESSING INTERACTIVE MULTIMEDIA 



INFORMATION OR SERVICES BY TOUCHING MARKED ITEMS ON 

PHYSICAL DOCUMENTS 

5 Field of the Invention 

The present invention relates to interactive hypermedia systems and more 
particularly to a method and system for creating hyperlinks from physical documents 
(manuscripts or printed documents) to locally or remotely accessible servers and for 
triggering said hyperlinks simply by touching marked items (e.g., v^ords, pictures, foot 
10 notes, symbols, icons) on said physical documents. 

Background of the Invention 

INTERNET 

The Litemet (the "Net") is a global network of computers and computer 
15 networks. The Litemet connects computers that use a variety of different operating 
systems or languages, including UNIX, DOS, WINDOWS, Macintosh, and others. To 
facilitate and allow the communication among these various systems and languages, the 
Intemet uses a language referred to as TCP/IP ("Transmission Control Protocol/Internet 
Protocol"). TCP/IP protocol supports three basic applications on the Intemet: 
20 transmitting and receiving electronic mail; logging into remote computers (the "Tehiet"); 
and transferring files and programs from one computer to another ("FTP" or "File 
Transfer Protocol"). 

WORLD WIDE WEB 

25 With the increasing size and complexity of the Intemet, tools have been 

developed to help find information on the network. These tools are often called 
navigators or navigation systems. Navigation systems that have been developed include 
standards such as Archie, Gopher and WAIS. The World Wide Web ("WWW" or "the 
Web") is a recent superior navigation system. 
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The Web is an Internet-based navigation system, an information 
distribution and management system for the Internet, and a dynamic format for 
communicating on the Web. 

The Web seamlessly, for the user, integrates format of information, 
5 including still images, text, audio and video. A user on the Web using a graphical user 
interface ("GUI," pronounced "gooey") may transparently communicate with different 
host computers on the system, different system applications (including FTP and Telnet), 
and different information formats for files and documents including, for example, text, 
sound and graphics. 

10 

HYPERMEDIA 

The Web uses hypertext and hypermedia. Hypertext is a subset of 
hypermedia and refers to computer-based "documents" in which readers move from one 
place to another in a document, or to another document, in a non-linear manner. To do 

15 this, the Web uses a client-server architecture. The Web servers enable the user to access 
hypertext and hypermedia information through the Web and the user's computer. (The 
user's computer is referred to as a client computer of the Web Server computers.) The 
clients send requests to the Web Servers, which react, search and respond. The Web 
allows client application software to request and receive hypermedia documents 

20 (including formatted text, audio, video and graphics) with hypertext link capabilities to 
other hypermedia documents, from a Web file server. 

The Web, then, can be viewed as a collection of document files residing 
on Web host computers that are interconnected by hyperlinks using networking protocols, 
forming a virtual 'Veb" that spans the Intemet. 

25 

UNIFORM RESOURCE LOCATORS 

A resource of the Intemet is unambiguously identified by a Uniform 
Resource Locator (URL), which is a pointer to a particular resource at a particular 
location. An URL specifies the protocol used to access a server (e.g., hyper text transfer 
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protocol, FTP, or other protocol), the name of the server, and the location of a file on that 
server. 

HYPER TEXT TRANSFER PROTOCOL 

5 Each Web page that appears on client monitors of the Web may appear as 

a complex document that integrates, for example, text, images, soxmds and animation. 
Each such page may also contain hyperlinks to other Web documents so that a user at a 
client computer using a mouse may click on icons and may activate hyperlink jumps to a 
new page (which is a graphical representation of another document file) on the same or a 

10 different Web server. 

A Web server is a software program on a Web host computer that answers 
requests fi'om Web clients, typically over the Internet. All Web servers use a language or 
protocol to communicate with Web clients which is called Hyper Text Transfer Protocol 
("HTTP"). All types of data can be exchanged among Web servers and clients using this 

15 protocol, including Hyper Text Markup Language ("HTML"), graphics, sound and video. 
HTML describes the layout, contents and hyperlinks of the documents and pages. Web 
clients when browsing convert user specified commands into HTTP GET requests, 
connect to the appropriate Web server to get information, and wait for a response. The 
response fi"om the server can be the requested document or an error message. After the 

20 document or an error message is retumed, the connection between the Web client and the 
Web server is closed. 

The first version of HTTP is a stateless protocol. That is, with HTTP there 
is no continuous connection between each client and each server. The Web client using 
HTTP receives a response as HTML data or other data. This description applies to 

25 version 1.0 of HTTP protocol, while the new version 1.1 breaks this barrier of stateless 
protocol by keeping the connection between the server and client alive under certain 
conditions. 
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BROWSER 

After receipt, the Web client formats and presents the data or activates an 
ancillary application such a sound player to present the data. To do this, the server or the 
client determines the various types of data received. The Web Client is also referred to as 
5 the Web Browser, since it in fact browses documents retrieved from the Web Server. 

INTERACTIVE ACCESS TO MULTIMEDIA SERVICES 

Interactive electronic services, video-on-demand, and the World Wide 
Web are providing access to an increasing offering of movies, shopping information, 

10 games, multimedia documents, electronic commerce and many other services. A major 
problem in using these systems is to browse the enormous variety and quantity of possible 
choices to discover what is available, and to make a selection. By example, when surfing 
on the Web, a conventional method to navigate across many pages of hypertext 
documents consists of using search tools or invoking bookmarked links to the different 

15 required topics. When surfing on video-on-demand services, a conventional method to 
navigate is to surf on channels. Advertisements on preview channels are used as entry 
points to other movies. Users can navigate and make selections from a remote control 
using hierarchical menus. Obviously, these approaches do not allow a rapid access and 
browsing of the thousands of multimedia documents that are available on the Web or 

20 interactive TV . 

The present invention is based on the recognition of two significant facts. 
The first fact is that people are very skilled at browsing through paper catalogs, 
magazines, newspapers, maps and books by flipping through the pages and glancing at 
pictures and text. 

25 A collection of printed color photographs can be much easily and quickly 

browsed than a sequence of computer screens. Paper has a nvmiber of usefiil properties: 
paper is easy to read, mark, and manipulate; and paper is portable, familiar and can be 
easily distributed. 
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Many electronic systems attempt to replace paper by providing many 
advantages such as, for example, a better access to multimedia services. However, most 
users prefer to work with paper. It is difficult to foresee, for example, the replacement in 
the future, of paper catalogs by electronic catalogs (e.g., by Web accessible catalogs). The 
5 publication entitled "The Last Book", IBM Systems Journal, Vol 36, No. 3 Vol 36, No. 3 
- 1997, by J. Jacobson, B. Comiskey, C. Tumer, J. Albert, and P. Tsao of the MIT Media 
Laboratory, compares printed books and computer screens in the following terms: 

"A book represents a fundamentally different entity than a computer 
screen in that it is a physical embodiment of a large number of simultaneous 

10 high-resolution displays. When we turn the page, we do not lose the previous page. 
Through evolution the brain has developed a highly sophisticated spatial map. Persons 
familiar with a manual or textbook can find information that they are seeking with high 
specificity, as evidenced by their ability to remember whether something that was seen 
only briefly was on the right side or left side of a page, for instance. Furthermore their 

15 haptic connection with the brain's spatial map comprises a highly natural and effective 
interface, when such information is embodied on actual multiple physical pages. 

Another aspect of embodying information on multiple, simultaneous pages 
is that of serendipity and comparison. We may leaf through a large volume of text and 
graphics, inserting a finger bookmark into those areas of greatest interest. Similarly, we 

20 may assemble a large body of similar matter in order to view elements in contrast to one 
another, such as might be done to determine which of a particular set of graphical designs 
is most satisfjdng". 

Out of those advantages the most important problem, of course, with 
traditional printed books is that they cannot be changed, amended, updated or completed. 

25 During the last years, due mainly to the widespread use of personal 

computers and the universal access of millions of users to the World Wide Web, the 
"multimedia publishing" has veritably exploded. Due to the widespread penetration of 
CD-ROM drives an enormous amount of multimedia titles combining text, images and 
sounds, are now accessible to owners of personal computers. In this evolution, an 
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incredible amount of hypermedia information is today accessible via the Intemet on the 
World Wide Web. 

Even when the public's enthusiasm for new computer-based multimedia 
services has been seen by many analysts as a threat to the conventional forms of 
hard-copied publishing, particularly book publishing, the real fact is that reading a book 
cannot be compared with reading an electronic media. Reading paper remains preferable 
for most people, whether they are familiar with computers or not. 

The second fact is that touching directly the objects we have around is one 
of the simplest, most instinctive, and universal human actions. Finger pointing and 
touching are the most natural form of human/machine interface. The action of touching 
is so simple and natural that navigating by means of touch screens require no training and 
no learning. In fact, the pervasiveness robustness and versatility of the "touch 
technology" is transforming the way people are living, working, learning, and playing. 
The "touch technology" is successfully used in many different applications. For instance, 
in industrial environments, environmentally-robust touch screens are increasing 
productivity under hazardous and hostile conditions that would cripple a standard 
Personal Computer and keyboard. In hospitals, touch input helps doctors to prescribe 
medications to patients faster by allowing handwritten prescriptions. In retail locations, 
interactive, through-the-window displays let customers shop whenever they want, even 
when a store is closed. In mobile and consumer devices, touch and stylus input is the 
widely accepted input method for portable, and other mobile devices. At tourist 
destinations, user-friendly kiosks are a cost-effective way to help travelers to get 
information and make their own reservations. 

The two main advantages of touching are simplicity, as touching with the 
fingertip is the simplest and more intuitive form of pointing and selecting an item, and 
versatility, as touching is particularly adapted to applications where the use of a keyboard, 
a mouse or an optical pencil or stylus is not practical or is not well adapted to the user's 
service or comfort. 
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Traditionally, a touch panel is integrated into the computer display. The 
touch panel and the display forms a combination called "touch screen". The input device 
is integrated into the monitor, so no space is wasted, and the interaction of the user with 
the system is made easier. The system guides the user by showing different choices in the 
5 form of icons displayed on the screen. When the user touches the icon of its choice, the 
associated action is executed. 

Therefore, there is a real need to provide the user with new systems and 
methods for improving printed texts with electronically stored data in the form of images, 
sounds and/or additional text. 

10 U.S. Patent 5,957,697, entitled "Printed Book Augmented with an 

Electronic Virtual Book and Associated Electronic Data", discloses a system and method 
for augmenting a printed text with electronically stored data in the form of images, 
sounds and/or additional text where a printed book comprising a pluraUty of pages of text 
is emulated by an electronic virtual book, that mimics the appearance of the printed book. 

15 This invention is based on the duplication of the paper book, on the creation of an 
electronic book over which links to hypermedia are defined and can be selected by the 
user. 

U.S. Patent 5,903,729, entitled "Method, System, and Article of 
Manufacture for Navigating to a Resource in an Electronic Network", discloses a method 

20 for navigating on an electronic network. The method comprises the steps of forming an 
electronic image of an object having a plurality of markable regions associated with a 
plurality of electronic resources; processing the electronic image to detect which of the 
markable regions associated with the plurality of electronic resources is marked; 
generating a list comprising at least one link to at least one of the electronic resources 

25 whose associated markable region is marked, and displaying a display screen based upon 
the list. However, this invention requires image scanning and image processing means to 
read marked regions on hard-copied documents. 

A similar approach, using optical image scanners to access multimedia 
services, is described in U.S. Patent 5,640,193, entitled "Multimedia Service Access by 
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Reading Marks on an Object". This patent discloses an apparatus and a method to enable 
a user to control the selection of electronic multimedia services by means of a scanner for 
reading marks on an object and for communicating a request signal, having an object 
code representing the read marks, to a user interface. 
5 U.S. Patent 5,495,581, entitled "Method and Apparatus for Linking a 

Document with Associated Reference Information Using Pattem Matching" also uses 
image scanning and pattem matching techniques. This patent discloses an apparatus for 
linking a portion of a document with associated reference information, wherein the linked 
portion is designated by a predetermined attribute of the received document image, using 

10 among several other, a device for electronically scanning the electronic representation of 
the document image to locate said predetermined attribute of the document's image. 

U.S. Patent 5,905,251 entitled "Hand-held Portable WWW Access 
Terminal with Visual Display Panel and GUI-based WWW Browser Program Integrated 
with Bar Code Symbol Reader in a Hand-supportable Housing" discloses a portable 

15 hand-held WWW access terminal for accessing HTML-encoded documents located on 
the WWW. The terminal includes a bar code symbol reader in a hand-supportable 
housing for reading URL-encoded symbols specifying the location of HTML-encoded 
documents stored in information servers connected to the Intemet and supporting the 
TCP/IP standard. This invention requires the marking of physical documents with bar 

20 code symbols and requires bar code readers to trigger hyperlinks. 

Finally, a different approach for having an access to multimedia services 
from physical documents is proposed in U.S. Patent 5,624,265, entitled "Printed 
Publication Remote Control for Accessing Interactive Media". This patent discloses a 
remote control system for an interactive media comprising a printed publication (such as 

25 a book, a magazine or a catalog), and one or a plurality of buttons physically attached to 
the printed publication to allow users to remotely control use of associated electronic 
content by a host device. 

From the prior art analysis, there is a real, uncovered need, for a new 
system and method of controUing the selection and the access to multimedia services 
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simply by touching with the fingertip items (i.e., words, icons, figures, foot notes, etc.) 
printed on books and more generally on any type of physical document. 

Summary of the Invention 

5 It is an advantage of the present invention that it improves the current 

systems and methods for selecting and accessing electronic multimedia services provided 
by one or a plurality of servers connected on a communication network. 

It is another advantage of the present invention that it can selects and 
access said electronic multimedia services directly fi*om physical documents. 

10 It is a particular advantage of the present invention that it can select and 

access an electronic multimedia service simply by touching a mark or a symbol 
highlighted in a physical docxmient. 

The present invention discloses a method of selecting and accessing 
information or service by touching marked items on a physical document, for use in a 

15 user system connected to a communication network comprising one or plurality of 
servers. The method comprises the steps of identifying a physical document; said 
physical document comprising one or a plurality of pages; identifying a page of said 
physical document; determining the position of a point pressed on a touch foil; said touch 
foil being placed and aligned over or under the identified page of the physical document; 

20 said page comprising one or a plurality of marked items; said touch foil being pressed at a 
point corresponding to a selected marked item; identifying the selected marked item 
referring to a hyperlink table, said hyperlink table comprising an indication of the 
position of each marked item on the identified page; identifying information or service 
associated with the selected marked item referring to said hyperlink table, said hyperlink 

25 table comprising for each marked item of each page of the document the identification on 
a server of the information or service associated with the selected marked item; and 
accessing the information or service associated with the selected marked item. 

The present invention also discloses a method of creating hyperlinks, by 
touching marked items on a physical document, for use in a user system connected to a 
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communication network comprising one or plurality of servers. The method comprises 
the steps of creating a hyperlink table for a physical document; said physical document 
comprising one or a plurality of pages; storing in said hyperlink table an identification of 
the physical document; for each page of said physical document, storing in said hyperlink 
5 table an identification of the page, storing in said hyperlink table an identification within 
the commimication network of information or service associated with each marked item 
and determining the position of points pressed on a touch foil; said touch foil being 
placed and aligned over or under the page of the physical document; said page comprising 
one or a plurality of marked items; said touch foil being pressed at points corresponding 
10 to marked item; and storing the position of the points pressed in the hyperlink table, said 
hyperlink table comprising for each marked item, an indication of its position on the 
page. 

The foregoing, together with other features and advantages of this 
invention can be better appreciated with reference to the following specification, claims 
15 and drawings. 

Brief Description of the Drawing s 

The novel and inventive features believed characteristics of the invention 
are set forth in the appended claims. The invention itself, however, as well as a preferred 
20 mode of use, further objects and advantages thereof, will best be understood by reference 
to the following detailed description of an illustrative detailed embodiment when read in 
conjunction with the accompanying drawings, wherein: 

Figure 1 shows the main components of the invention in accordance with a 
25 preferred embodiment of the present invention; 

Figure 2 shows an example of touch foil technology in accordance with a 
preferred embodiment of the present invention; 
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Figure 3 shows the process of selection and access to Web pages from 
items marked on a physical document in accordance with a preferred embodiment of the 
present invention; 

Figure 4 shows a physical document; 
5 Figure 5 shows how items to access information on servers are marked in 

accordance with a preferred embodiment of the present invention; 

Figure 6 shows how touch foil is placed on a document in accordance with 
a preferred embodiment of the present invention; 

Figure 7 shows how the user presses the touch foil over marked items in 
10 accordance with a preferred embodiment of the present invention; 

Figure 8 shows how to use the touch foil to enter a document reference 
number in accordance with a preferred embodiment of the present invention; 

Figure 9 shows how to use the touch foil to enter a document's page 
number in accordance with a preferred embodiment of the present invention; 
15 Figure 10 shows how the user presses the touch foil over selected item in 

accordance with a preferred embodiment of the present invention; 

Figure 1 1 shows how the information linked with the item touched on the 
document is shown in accordance with a preferred embodiment of the present invention; 

Figure 12 is a flow chart of the method of creating hyperlinks on a 
20 physical document in accordance with a preferred embodiment of the present invention; 

Figure 13 is a flow chart of the method of triggering hyperlinks on a 
physical document in accordance with a preferred embodiment of the present invention; 
and 

Figure 14 shows the Miami MetroMover paper map placed underneath a 

25 touch foil. 

Detailed Description of the Invention 

The present invention discloses a system and method for selecting and 
accessing multimedia information and/or services located on one or a plurality of servers 
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connected to a communication network simply by touching with a finger items marked on 
a physical document or on any other physical surface. The system includes a touch foil 
placed over (or under) the document for reading coordinates of selected marked items, 
and an user workstation for accessing the information and/or the service associated with 
5 the selected marked items. 

Items marked on the documents are selected by pressing the touch foil 
placed over (or under) the document. Once a marked item is selected, the user 
workstation receives from the touch foil a signal indicating the position of the selected 
marked item. The user workstation identifies in a hyperlink table a server and, within this 

10 server, the information and/or the service associated with the position of the selected 
marked item. Finally, a request is sent to the identified server for accessing the desired 
information and/or service. 

In a particular embodiment, the user workstation is connected to the 
Intemet network and comprises a Web Browser application. Servers are Web servers and 

15 the information or/and the service are Web pages linked to the items marked on the 
physical document and selected by the user when pressing the touch foil on the 
corresponding mark on the document. 

SYSTEM FOR SELECTING AND ACCESSING MULTIMEDIA INFORMATION 

20 As shown in Figure 1, the system according to the present invention 

comprises a hard-copy document 101 (e.g., a book) with printed marks identifying links 
to multimedia information and/or services on servers, a touch foil 102 (to be placed over 
or undemeath a page of the document), and a user workstation 103 for accessing and 
displaying multimedia information and services. 

25 

Physical Document 

The physical document 101 can be of any kind, for example, a newspaper, 
a geographic map, a novel book, a text book, a technical book, a commercial catalog or 
even any other type of engraved or printed surface (e.g., a painting in a museum of art). 
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The material of the document can be paper, plastic, wood or other material. The marks 
identifying links from some items of the document to multimedia information or services 
on servers can be printed marks placed by the document's author or they could be any 

kind of mark of annotation written by the user. For example, each photograph or 
5 advertisement in a newspaper, magazine, or catalog can be accompanied by a printed 
mark. 

Touch Foil 

The touch foil 102 may be made of transparent resistive or capacitive films 

10 of the type used commonly to manufacture touch screens. The generated signal is 
generally proportional to the coordinates of the point pressed. One example of touch foil 
it would be possible to use is the TouchTek4 (4-wire Analog Resistive Touchscreens) of 
MICRO TOUCH company (TouchTek™ is a trademark of the MICRO TOUCH 
Company). The TouchTeck4 technology is fully described in the Web site of MICRO 

15 TOUCH Company at http://www.microtouch.com/. 

As illustrated in Figure 2, TouchTek4 touch screens 210 use a polyester 
sheet 201 with a conductive coating 202 as a vertical conductive electrode, providing 
one-half of a touch coordinate. The topsheet 200 is separated from the bottom layer, a 
glass substrate 204 with a conductive coating 203, by spacer dots 205. The bottom sheet 

20 206 is attached to the top sheet 200 by means of an adhesive 209. The bottom sheet 206 
forms the horizontal conductive electrode, generating the second half of the touch 
coordinate. Silver bussbars located on the topcoat 207 and on the bottom sheet 208 pass 
the touch coordinates to the touchscreen controller. TouchTek4 touchscreens feature 
hard-coated polyester topsheets 200, available in several surface finishes. Spacer dots 

25 205 are available in several dot arrays, optimized for finger, pen and finger, or pen-only 
input. The conductively-coated glass bottom sheet 203, 204 is available in several 
thicknesses. Electronic control is provided by a serial controller, or by 4-wire controllers. 
TouchTek4's specifications include narrow inactive border areas and compact touch 
sensors which allow system designers and OEMs to provide the largest useable screen 
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area and full mouse emulation without sacrificing functionality or consuming excess 
power. Apart from being suited for implementing the functions of this invention, 
TouchTek4 touchscreens are commonly used in hand-held personal information 
management systems, PDAs, mobile computing systems, automotive, diagnostics and 
5 telecom devices, and Internet appliances. TouchTek4 touchscreens are engineered to 
accept more than three million touches to any area of the screen. 



User Workstation 

The user workstation 103 is used to access information and/or services 
10 located on servers connected on the network. The user workstation may be, for example, 
an Litemet appliance, a multimedia Personal Computer (PC), a set-top box, a TV 
monitor, a game console ... The touch foil may communicate with the user workstation by 
means of a cable, a wire pair, an infrared link, or a wireless radio Imk. 



15 Information and/or Service Access 

Li order to retrieve and display a multimedia information or to access a 
multimedia service associated with a marked item on a document 101, the user touches 
with his or her finger or exercises a pressure on the portion of the touch foil 102 placed 
over (or under) the marked item he or she wishes to select. The position of the marked 

20 item selected on the touch foil identifies the server and the information within this server 
the user wants to access. The user workstation 103 then accesses the identified server and 
retrieves and displays the information or/and service associated with the selected marked 
item. 



25 WORLD WIDE WEB 

Li the particular embodiment of the invention shown in Figure 3, the user 
workstation 300 is connected to the Intemet network 301. The user workstation 
comprises a user interface 302 including a Web Browser 303 (also called "Web CUent") 
to access the World- Wide- Web (WWW). In order to access the Web pages associated 
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with a marked item on a document 309, the user touches with its finger 306 the touch foil 
307 placed over or under the marked item 308 he wishes to select. The position of the 
marked item selected on the touch foil identifies the Web server 305 and the Web pages 
the user wants to retrieve and display. The Web Browser program 303 sends an HTTP 
5 request 304 to the identified Web Server 305. The response to the request (HTTP 
response) is sent by the Web Server 305 in the reverse direction to the Web Browser 303. 
The HTTP response comprises the requested Web pages associated with the selected 
marked item. 

10 SELECTION AND ACCESS TO INFORMATION OR/AND SERVICES ON 
SERVERS 

A) Creating Hyperlinks on a Physical Document 

As shown in Figure 12, the method for creating hyperlinks from a hand 
written or printed document (like the document shown in Figure 4 entitled "Early British 

15 Kingdoms"), to a plurality of servers to access multimedia information or services, 
comprises the steps of: assigning a reference number (identifier) to the doctmient (step 
1201); for each page of the document (or portion of the document): marking items 
(hyperlinks) on the page (step 1202); creating hyperlinks between these marked items and 
information and/or services located on servers (step 1203); placing and aligning a touch 

20 foil over (or under) the page (step 1204); pressing the touch foil on marked items on this 
page (step 1205); reading and storing in a hyperlink table the coordinates of marked items 
on this page (step 1206). 

Assigning a Reference Number to the Document 

25 As shown in Figure 4, for each document he receives, the user assigns a 

reference number (identifier) to this document (e.g., 387) for identifying said document, 
writes this reference number (identifier) on the document, and creates a hyperlink table 
associated with the document and accessible from the workstation, said hyperlink table 
comprising the reference number of the document and other relevant information related 
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to the document such as title, author, ISBN (International Standard Book Number), or 
date. The hyperlink table may be stored within the user workstation or may be stored in 
an external memory accessible from the user workstation. 

The header of the new created hyperlink table associated with the 
5 document shown in Figure 4, can be built as follows: 



Doc: 0387 


Title: "Early British Kingdoms" 


Autiior: David Nash Ford of Binfield, Berkshire, UK 




Date: 28/01/2000 


ISBN: 84-344-0856-2 


Pg: 






X= |Y= 


Link: 





Marking Items on Pages of the Document and Creating the Hyperlinl^s between 
10 tliese Items and information and/or services located in Servers 

As shown in Figure 5, each time the user wants to create a hyperlink for an 
item on a page of a document, he marks the item on the page of the document (e.g., by 
underlining it), enters the page number where the item is marked in the hyperlink table of 
the document (e.g., 16), associates this item with a destination address within the 
15 communication network, this destination address identifying a server connected to the 
commimication network and the information and/or services within this server the user 
wants to access, and enters the destination address associated with this item (e.g., an URL 
address for example) in the hyperlink table. 

20 The hyperhnk table associated with the page shown Figure 5, can be built as follows: 



Doc: 0387 


Title: "Early British Kingdoms" 


Author: David Nash Ford of Binfield, Berkshire, UK 




Date: 28/01/2000 


ISBN: 84-344-0856-2 


Pr: 16 






X= 


|Y- 


Link: Mabinogion 


http ://www.cyberphile.co, uk/~taffi'taffiiet/mabinogion/mabinogi 

on. htm 


X= 


|Y= 


Link: Avalon 


http://freespace.virgin.net/david.ford2/avalon.html 


x= 


|Y= 


Link: Afallach 


http://freespace.virgin.net/david.ford2/gods.html#Afallach 


x= 


|Y= 


Link: Bell Mawr 


http://freespace.virgin.net/david,ford2/gods.html#Beli 



25 
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Placing and Aligning the Touch Foil over a Page of the Document 

As shown in Figure 6, after the hyperlinks of a page have been defined in 
the hyperlink table associated with the document, the touch foil is placed over (or under) 
the page, and aligned with the borders of this page by some conventional means (e.g., by 
5 adjusting the upper left comer of the touch foil with the upper left comer of the page). 

Reading the Coordinates of Hyperlinked Items on the Page 

As shown in Figure 7, for each item previously marked on the page 
C' Mabinosion '\ '' Avalon '\ "AfallacK \ ''BeliMaw£'), the user presses (e.g., by the finger 
10 tip) the touch foil on the corresponding mark to determine the position (for instance, the 
coordinates) of the item on the page. 

Once the measure of the coordinates of each marked item on the page is 
completed, the hyperlink table is updated as follows: 



15 



Doc: 0387 


Title: "Early British 
Kingdoms" 


Author: David Nash Ford of Binfield, Berkshire, UK 




Date: 28/01/2000 


ISBN: 84-344-0856-2 


Pg: 16 






X=30 |Y=95 


Link: Mabinogion 


http://www.cyberphile.co.uk/~taflC^tafFnet/mabinogion/niabinogion. 
htm 


X=255 |Y=150 


Link: Avalon 


http ://freespace . virgin .net/david . ford2/a valon . html 


X=225 |Y=160 


Link: Afallach 


http://freespace.virgin.net/david.ford2/gods.html#Afallach 


X=190 |Y=230 


Link: Bell Mawr 


http://freespace . virgin .net/david .ford2/gods.html#Bel i 



For example, page 16 of document 387 entitled "Early British Kingdoms" 
written by David Nash Ford of Binfield, the coordinates of the underlined word 
''Mabinogion"' are X=30 / Y=95. This underlined word ' 'Mabinomon' ' points to the URL 
20 address http://www.cyberphile.co.uk/-'taf£^taffiiet/mabinogion/mabinogion.htm. 

Using the herein described method for the different pages of a same 
document, the hyperlink table would appear like this: 

25 
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Doc: 0378 


Title: "Early British Kingdoms" 


Author: David Nash Ford of Binfield, Berkshire, UK 




Date* 28/01/ 2000 


ISBN* 84-344-0856-2 


Pg:3 






X=45 |Y=130 


Link: Buellt & Gwerthrynion 


http://freespace.virgin.net/david.ford2/buellt.html 


X=205 IY=170 


Link: Caer-Baddan CBath) 


http://freespace. virgin, net/david.ford2/south.html#Gloui 


X~1S IY=190 


Link* Paer-Pelemion TSilchester^ 


http i/Zfreespace. virgin, net/david . ford2/vorti gem.html 


X=110 IY=255 


Link: Ceredigion 


http://freespace.virgin.net/david.ford2/ceredigion.html 














Pp* 16 








ijiujVa ivtauitiiiigii/ii 


http://www.cyberphile.co.ulc/'^taff/taffnet/niabinogion/mabinogi 
on. htm 


X=255 |Y=150 


Link: Avalon 


http://freespace.virgin.net/david.ford2/avalon.html 


X=225 |Y=160 


Link: Afallach 


http :// freespace. virgin .n et/david . ford2/gods .html# Afallach 


X=190 |Y=230 


Link: Bell Mawr 


http ://freespace. virgin .net/david . ford2/gods.html#Beli 














Pg: 39 






X=25 1 Y=30 


Link: St. Joseph of Arimathea 


http://freespace.virgin.net/david.ford2/joseph.html 


X=80 1 Y=75 


Link: The Kings of Dumnonia 


http://freespace.virgin.net/david.ford2/dumnonia.html 


X=140 |Y=135 


Link: Eudaf Hen & Conan Meri. 


http://freespace.virgin.net/david.ford2/eudanc.html 









B) Triggering Hyperlinks from a Physical Document 
5 The user receives a hand written or printed document, like the document 

entitled "Early British Kingdoms", (Doc: 387) shown in Figure 4. Hyperlinks have been 
previously defined between items marked on the pages and information and/or services 
on servers connected on the communication network. As shown in Figure 13, the method 
for triggering hyperlinks and to access information and/or services directly from the items 

10 marked on the pages of the physical docxmient comprises the steps of: selecting a 
document by entering the reference number of this document (step 1301); selecting a 
page comprising one or a plurality of marked items (step 1302); placing and aligning a 
touch foil over (or under) the selected page (step 1303); selecting a marked item on this 
page by pressing the touch foil on the corresponding mark (step 1304); determining the 

15 position of the point pressed on the touch foil (step 1305); identifying the selected marked 
item referring to a hyperlink table, this hyperlink table comprising an indication of the 
position of each marked items on the selected page of the document (step 1306); 



FR920000015US2 



identifying the information or/and the service associated with the selected marked item 
referring to the hyperlink table, this hyperlink table comprising for each marked item of 
the selected page the identification of the requested information and/or service within the 
network (preferably by means of a destination address) (step 1307); accessing the 
5 information and/or service (step 1308); retrieving and displaying this information and/or 
service (step 1309). 

Entering the Reference Number of the Document 

By means of any user interface (keyboard, mouse, touch screen ...), the 
10 user enters the reference number (identifier) of the document (e.g., document 387) he 
wants to select. In the particular embodiment shown in Figure 8, the user enters the 
reference number of the document (e.g., Doc: 387) by means of the array of pressure 
sensitive "touch buttons" printed on the top of the touch foil. The user presses in the 
following order: a touch button marked "Doc," and then numerical touch buttons 
15 corresponding to each digit of the document reference number (e.g., "3", "8" and "7"). 
This procedure gives access to the hyperlink table associated with this selected document. 

Selecting a Page Comprising a hyperlink 

By means of any user interface (keyboard, mouse, touch screen ...), the 
20 user enters the page of the document (e.g., page 16) (or the portion of the document) he 
wants to select. In the particular embodiment shown in Figure 9, by means of the array of 
pressure sensitive touch buttons printed on the top of the touch foil, the user enters the 
page number to select (e.g., Pg. 16). The touch foil can be on any position (normally the 
touch foil is placed over the page comprising the hyperlink to trigger). The user presses 
25 in the following order: a touch button marked as "Pg", and then numerical touch buttons 
corresponding to each digit of the page number (e.g., "1" and "6" ) to select. This 
procedure gives access to the selected page (e.g., Pg. 16) within the hyperlink table 
associated with the selected document (e.g., Doc: 387). 
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Placing and Aligning the Touch Foil over or under a Page of the Document 

After having selected a document and a page in this document, the touch 
foil is placed over or under the page, and aligned with the borders of the selected page by 
some conventional means (e.g., by adjusting the upper left comer of the touch foil with 
5 the upper left comer of the page). 

Selecting a Marked Item on this Page 

As shovm in Figure 10, The user selects a marked item (A fallach) on the 
page by pressing (e.g., by means of his fingertip) the touch foil on the corresponding 
10 mark. 

Determining the Position on the Page of the Point Pressed on the Touch Foil 

The touch foil sends a signal to the user workstation to identify to selected 
marked item. This signal indicates the position on the page of the point that has been 
pressed by the user on the touch foil. The generated signal is generally proportional to the 
1 5 coordinates (X/Y) of the point pressed. 

hi our example, the touch foil measures the position on the page of the 
point pressed C 'A fallach' ') by the user. The coordinates that are measured at this point are 
around X=225 and Y=l 60. 

20 Identifying the Selected Marked Item 

The marked item selected on the touch foil by the user is identifying 
thanks to the hyperlink table, said hyperlink table comprising an indication of the position 
(coordinates X, Y) of each marked items on each page of the document. 

In our example, the coordinates measured by the touch foil are around 
25 (close to) X=225 and Y=160). They corresponds in the hyperlink table to the marked item 
"Afallach' \ 

The hyperlink table is either stored locally in the user workstation, or is 
retrieved fi:om a remote server and then stored locally in the use workstation or is stored 
in a remote server and is accessed remotely . 
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Identifying the Information Associated with the Selected Item 

The hyperlink table comprises for each marked item of each page of the 
document the locaHzation within the network of the requested information and/or service. 

The information and/or service may be localized by means of a destination address. In 
5 the Intemet network, Web pages in Web Servers are identified by an URL (Uniform 
resource Locator). 

In our example, the hyperlink table associates the marked item ^ 'A fallach' ' 
with the URL : http://freespace.virgin.net/david.ford2/gods.html#Afallach. 

10 Accessing the Information and/or Service 

The user workstation Web triggers the hyperlink (destination address, 
URL, ...) associated in the hyperlinks table, with the identified marked item. 

In our example, the hyperlink labeled ' 'Afallach " is triggered since the 
system determines from the hyperlinks table that, for this page (i.e., Pg. 16), X=225, 
15 Y=160 are the coordinates of the nearest hyperlink to the sensed position. Thus, in this 
example, simply pressing near the marked item ' 'A fallach " will automatically trigger the 
following hyperlink on the Web: 

http://freespace.virgin.net/david.ford2/gods.html#Afallach 

20 Retrieving and Displaying the requested Information and/or Service 

The information contained at the selected destination address (URL) can 
be displayed on the user workstation. As shown in Figure 11, additional information 
(e.g.. Document number, page number, marked item, foil coordinates and URL) related to 
the marked item selected by the user can be shown on the Web Browser along with the 
25 information retrieved from the Web Server. 
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ALTERNATIVE APPLICATIONS 

Other several possible applications of the present invention are described 
below. Each of these applications basically use the same previously described method 
and system. 

5 

Installation and Engineering Instructions 

A customer receives a complex computer equipment, with an installation 
manual comprising drawings and schemes of the parts and subassemblies. Theses parts 
and subassemblies are labeled with printed marks representing hyperlinks for accessing 

10 additional information on a remote Web server. When the customer touches one of those 
marked items, multimedia instructions to show how the part needs to be installed or 
serviced are instantly displayed. It is no more necessary to look through printed manuals 
to discover how a part is called and what to do with it. Furthermore, the customer does 
not need anymore to navigate on a computer terminal among different alternatives paging 

15 through multiple menus and choices and loosing the attention on the general picture and 
friendliness of the installation manual. A single printed copy of a general view of the 
equipment is sufficient to navigate with the system according to the present invention. 
The customer has just to press with his finger on the desired item over the physical of the 
installation manual. 

20 

Newspapers and Magazines 

A subscriber reading a newspaper or magazine, may be interested in seeing 
computer multimedia or TV video information associated with the articles he reads. 
While reading the sports pages (e.g., on the New York Times), key events can be instantly 
25 recalled and played on demand (e.g., the opening ceremony of Melboume Olympic 
Games, the last images of the "Tour de France," or the last tennis match on Winbledon) 
simply by touching a mark, an icon, or an underscored or bold printed word on pages of 
the newspaper. 
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Paper-based Advertising Linked to On-demand Multimedia Ads 

Today, many free-toll calls originate from people reading advertisements 
in newspapers or magazines or in direct mail ads. According to the present invention, 
people can instantly access multimedia presentations simply by touching the ads that have 
5 drawn their attention. 

Courses and Textbooks 

Extensive reading is easier to do on paper, but animated video 
explanations and demonstrations are much more effective for some purposes. The two 
10 can be tied together by placing hyperlink marks in a textbook. These hyperlink marks 
can, for example, link the textbook to live discussion groups with other students or to live 
interactions with professors and tutors. 

PARTICULAR APPLICATION 
15 Hypermedia Access from a Paper Map 

In the particular example shown in Figure 14, the map (e.g., a paper map 
of the MetroMover, Miami, Florida) is placed and aligned over the touch foil. By 
pressing the map over any selected station name (small circles such as "Freedom Tower," 
"Financial District," or "Government Center"), the information on the Web associated 
20 with this foil position (i.e., the information related with the station selected on the map) is 
retrieved from the Web and displayed on the user Web Browser located in its 
workstation. 

What has been described is merely illustrative of the application of the 
principles of the present invention. Other arrangements and methods can be implemented 
25 by those skilled in the art without departing from the spirit and scope of the present 
invention. 
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